An act, process, or methodology of making something (such as a design, system, or decision) as fully perfect, functional, or effective as possible. – Merriam-Webster
Code optimization is the process of enhancing code quality and efficiency.
“Make it work, then make it beautiful, then if you really, really have to, make it fast. 90 percent of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!” –Joe Armstrong
Optimization should be the final step of your programming practice. Performance can be high following the pre-optimization steps. Only optimize when it’s necessary!
{profvis} and {profile}.{microbenchmark} and {bench}.{memoise} (non-persistent by default) and {R.cache} (persistent). {snow} and {parallel}.{ff}, {bigmemory}, and {feather}.gc() to release memory (not needed, but it doesn’t hurt to use after removing large objects).{data.table} for faster computations.Goal: build a model using the insurance claims of ~25 million lives with two years’ worth of data (i.e., billions of records!) to estimate the cost of a procedure.
Steps:
Took months to complete…
…Other solutions are also possible!
# R Studio Server: 420ms
# R Studio Desktop Below
library(profvis)
library(git2r)
profvis({
git_object <-
function(data_object = NULL) {
object_names <- sort(unique(subset(odb_blobs(), grepl(".rda", name))$name))
git_obj <- grep(data_object, object_names, value = TRUE, ignore.case = TRUE)
return(git_obj)
}
git_object("pkg")
})Unit: milliseconds
expr
git_object <- sort(unique(subset(odb_blobs(), grepl(".rda", name))$name))
git_object2 <- sort(unique(gsub(".*/", "", system2("git", "ls-files *.rda", stdout = TRUE))))
min lq mean median uq max neval cld
3232.7150 3699.6561 3977.4452 4029.9945 4328.2491 4557.8492 10 a
233.1812 235.5819 249.6366 239.2543 243.3913 346.6082 10 b… using base::system2() produces much faster results than git2r::odb_blobs(). Sometimes, what you want to use isn’t always the best option for the end user!
“The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.”
- Donald Knuth, Computer Programming as an Art
TIME!